Improved probabilistic prediction of healthcare performance indicators using bidirectional smoothing models

Smoothing of observed measures of healthcare provider performance is well known to lead to advantages in terms of predictive ability. However, with routinely collected longitudinal data there is the opportunity to smooth either between units, across time or both. Hierarchical generalized linear models with time as a covariate and hierarchical time series models each result in such two-way or ‘bidirectional’ smoothing. These models are increasingly being suggested in the literature, but their advantages relative to simpler alternatives have not been systematically investigated. With reference to two topical examples of performance data sets in the UK, we compare a range of models on the basis of their short-term predictive ability. Rather than focusing on point predictive accuracy alone, fully probabilistic comparisons are made, using proper scoring rules and tests for uniformity of predictive p-values. Hierarchical generalized linear models with time as a covariate were found to perform poorly for both data sets. In contrast, a hierarchical time series model with a latent AR(1) structure has attractive properties and was found to perform well. Of concern, however, is the large amount of time that is needed to fit this model using the WinBUGS software. We suggest that research into simpler and faster methods to fit models of a similar structure would be of much benefit.